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A unique feature of the Research on School 
Effectiveness Project (part of a larger program for school 
effectiveness initiated by the State of Alaska) is an intention to 
base the development of new standards for effective schooling upon 
documented research f indings. First, a list of variables, affecting 
some part of the educational community was generated; some were 
changed to correspond more closely to the descriptors used in the 
ERIC system. Next, the process of identifying and retrieving 
appropriate materials was facilitated by the Alaska Department of 
Education's contract to use tha services and resources available 
through the Northwest Regional Education Laboratory (NWREL) 
Information Center, a special purpose library with a computer 
terminal for direct apcess to the ERIC files. and other data bases in 
the Lockheed Inf donation System. Sample "Item Report* and "Summary 
Report by Category" forms are included to show how material 
identified for its usefulness and quality of study is analyzed. An 
information flow chart depicts the search, analysis, and synthesis 
procedure. Appendices include formats of the two forms used, a list 
of standards for rating studies, and a sample copy of a decision 
display. (ERB) * . » 
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Introduction 

Che Research, on Scho'dl Effectiveness Project? is a part of a larger 
program for school ^fttetiveness initiated by the S.tate of Alaska.^ It 
is expected that the overall program will result in, the generation x>f 
state standards for effective schooling and, ultimately, methods and 
procedures for monitoring and evaluating compliance* with those standards, 

A 1 unique feature of the Research on School Effectiveness Project is 
an intention* to base the development of new standards for effective schooling 
upoh documented research findings. It is held that, if we,^as a community 
of professional educators and concerned, informed' citizens , ds know^ with 

r . . 

reasonable confidence that certain educational practices are effective, 

we are bound to put them into action as fully as possible. On th« other hand, 

if there are educational practices for which we tan find no evidence, or 

CI ■ J 

only weak, contradictory or inconclusive evidence of effectiveness, then 
we have no justification to require or even strongly suggest that they be 
used, no matter how appealing or logical they may seem to an interested 
proponent. * 1 * 

Local options and local educational decision making will be preserved. 
The new standards will be suggested options which^Uie local community of 
♦citizens, advised by their professional educators, canno^ignore No 
coercion is intended, except for whatever coercion is inherent in documented 
evidence. Research, especially educational research, never proves 

i - '■ . 1 

anything, but 1 it may at times produce evidence that is strongly suggestive — 

\ 

so strong that it cannot reasonably be ignored. The Searching out of these 
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strongly ^suggestive pieces of educational research is the object of this 
present project. It is a prelude to,, and a necessary steR which must be 
taken- before the larger step of Actually developing and selecting standards 
The Research on School Effectiveness Project is not % seen as a limited, 
one-time effort. The body fcf documented educational research is-vast and 
it is, of course, being added to daily. It is intended that the project 
be continued into the reasonable future for as long as-it, yielda useful 
information. for policymaking on school standards and practices. The first 
p^ase of the project is, understandably, .of limited duration and depth. 
The Alaska Department of Education contracted with the Northwest Regional 
'Educational laboratory _*to conduct a preliminary search of the research 
literature and, perhaps- more importantly, to develop methods ar*dj procedures 
whereby such a ^fearch can be conduced systematically , 'continuously, 
ecpnomicallv, and with the feature of possible replication, so that 
findings can be challenged", changed, or otherwise modified if better 
evidence becomes available. This initial; phase will occupy approximately 

two| months, -November and December of 1980, and will culminate with a'set 

1 

of reports oneffective school practices „to the Governor's Task Force on 
Effective Schooling, plus this present document as a- recommended pro- 
Cedure.to extend the search for documented evidence on educational 
practices which really work. 

On November 4/1980 a meeting was held in Juneau betweep a design 
group representing the Department of Education and the contractor for 
th£ preliminary study, the Northwest Regional Educational Laboratory, 
Division of .Evaluation, Research and Assessment. A list* of possible - 



variables to be investigated was generated by the Department of Education 
design group, and a set of approaches and procedures to be used in the ' 
investigations-was presented by the contractor. Mutual agreement was .reached 
on both the variables and ^the procedures. The contractor was given the * 
tasks of further refining arid defining .the variables and procedures and of 

■ ■ * ■ i *> ' ° 1 • / 

initiating a search of the literature using the new procedures. The - / 

! 

remainder fcf this paper will consist of an exposition of the list of ^ 
variables^and an outline of the procedures which are followed fti the 
search and analysis df- the literature- 



Variables to be -Inve stigated ' 

n; J * 1 , 

The" original list of variables (categories) generated at the NovemBer 4 
meeting, is included in the appendix. That list was subsequently expanded 
slightly, arid more importantly, was par tit ioned Anto two lists of variables: 
one group which can, under certain circumstances, be modified; and .th£ 
other group, which usually must be considered as given. A few of the 
variables appear on both lists. For example, teacher characteristics in 
a school can be modified over time by selective hiring, trahsfering md 
training; but in the short run, because Qf tenure considerations, 

-I 

difficulty, in^ recruiting and otirer factors, must often -be considered as 

givens in a particular situation. 

/ 

The variables which may be .controlled or, at least, strongly ' affected 

by some part of the educational community, and the variables which must, 

at least under certain circumstances, be taken as given, follow as Charts 1 

and 2, respectively. Both lists are subject to further modification. In 

I 

addition, the/labels for some of the items have be-on ^hanged to make them 



correspond more closely to the descriptors used <n the ERIC syst 



em. 



CHART 1 



VARIABLES WHICH MAY BE CONTROLLED OR AT LEAST STRONGLY AFFECTED BY SOME 
/ • • PART OF THE EDUCATIONAL COMMUNITY " 



Original Descriptors 



ERIC Descriptors 



• K 

ERJC 



• Class Size 
School Size * ' 
Physical Facilities* 
Provision of Materialg/Equipraenr/ 

Technology 
Textbooks - 

Media Centers /Audiovisual Equipment 
Computer Assisted/Managed Instruction 
Libraries 

Nutrition . ' 

General Leyel of Financial Support' 
Teaching Strategies/Instructional Practices 
Differentiated Staff (including use of 

aides-) , • f * 

Individualization^ 
Diagnosis /Prescription 
\ Use of Required Homework 
Pupil Grouping ■ » * y 

Bilingual Approaches 
Time on Task . fc 

Instructional Leadership 
^ble of Principal 

• Other Instructional Leadership 
Administrative Practices /Policies 
Teacher Evaluation 

School "Climate" ' 
Student Activities 

Existence and Type of Parent Involvement 
Programs 

Staff -Development a»d Inservice Training 

Programs 
Existence of Preschool Programs 
' Attendance Improvement* Programs 
Teacher Characteristics 
Teacher Certification, 
Teacher Competence in Content 

Teacher Training in Pedagogy 



Class Size t ... 

School Size 

Facilities 

Media Selection/Educational 

Technology - " 

Textbooks 

Learning Resource Centers/ 
Audiovisual Aids 

Computer Assisted Instruction/ 1 
Computer Managed Instructir- 
Schoo^Libraries 

Nutrition % m « 
Financial Support * 
Teaching Methods 
Dif ferent'iated'Staffs " * 

Indiyidualiz'ed Instruction 
Diagnostic Teaching 
Homework x 

.Grouping (Instructional Purposes) 

Bilingual Education 

Time Factors (Learning 
' filass Organization 

Instructional 'Developmen£/Lead£rship 

Pr incipals/ Ins t rue tional Development 

Instructional Development /Leadership 

Administrative Policy 

Teacher Evaluation 

Educational Environment 

Extracurricular Activities 

Parent Participation 

* « 

Inservice Teacher Education 

Preschool Education 

At t endance'/ Improvemen t Programs ' 

Teacher Characteristics \ 

Teacher Certification 

Teacher Qualifications/Intellectual 

Disciplines 
Teacher education/ Instruct ion 
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CHART 2 



VARIABLES WHICH MUST (AT LEAST UNDER CERTAIN CIRCUMSTANCES) 

BE TAKEN AS GIVEN 



Original Descriptors 



ERIC Descriptors 



Student Charact ristics 
Ability , ' 

Socioeconomic Status 
Language Background 
Subject Matter 
Grade Level 
Home Enviroriment 
Attendance 

Degree of Parent/Community 

Involvement „ ' 
Teacher Characteristics 
Teacher Certification 
Teacher Competence in Content 

Teacher Training in Pedagogy 



Student Characteristics 
Academic Ability - 
Socioeconomic Status 
Chil^Language 

Instructional Program Divisions 
Family Environment * 
Attendance 

Parent Participation/CrMJwnurkity 

Involvement • « 4 

Teaqher Characteristics 

Teacher Certification fc 

Tedcher Qualifications /Intellectual 
Disciplines 

Teacher Education/ Inst ruction 

Teachet Behavior 

Teacher Attitudes 
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Chart 1 migftt^ be thought of as tl?e vertical axis of a matrix*, with Chart 2 

r 

describing the items on the horizontal axis. Such a matrix might be used 
later as a framework for analysis anS subsequent synthesis. 

■ ■ . , r . 

Se ? arch, Analysis and Synthesis Procedure ^ % \ 

V. 

The process pf identifying and retrieving appropriate materials is 
facilitated by the services and resources available through t Ae NWREL - 
Information Center, a modern, special purpose l^ftrary with, a computer 
terminal for direct excess to the ERIC' files and other data bases in the 
Lockheed Information System. In addition to an ERIC microfiche collection and 
a limited number-, of ls4sic books on educational topics, this library hou'ses 
an extensrve pamphlet file with cross references to'others pf its holdings, - 
preference book collection and a collection of publications from ttte ERIC 
clearinghouses, and subscribes to approximately 350* educational joifrnals 
and newsletters. Interlibrary loan services are available, and as library 

r 

staff travel to nearby Portland State University twio^we^ly to borrdfc 
ot photocopy materials, project staff have assistance in retrieving needed 
items. Searches of ERIC and other computerized ,data bases ^Are provided. , 
by the library's information retrieval specialist. 

% Two additional library services support tb£~Vfforts of project staff 
to access newly published materials. One is pe practice of scanning 

all periodicals a's th<y arrive and alerting staff as articles and reviews 

v 

of books in their areas of int'erest appear. The other is that all materials 

\ J 

purchased by NWREL programs .are order* i and processed through the library, 

* 

which enables library staff^trTi) inf requesters when materials they 
wish to purchase are already owned ^y another, program, a time- and money- 
saving prac ice; and 2) routinely refer staff of different programs to one 
another's resource "collections when relevant. 

8 




In addition to library services and resources, project staff receive 
assistance in locat^nlfmlit^ other WREL programs and 

from people in NWREL management positionV'Sho have taken an interest in 
the project, as aell a's examining resources available through the Alaska 
Department of Education's Office of Planning and Research. 

THis combination of formal* institutional information services^and 
% collegial support of the proj^c^'s efforts enables project staff to identify 
and retrieve materials in a timely and economical manner. 
J . Tne search, ^analysis and synthesis procedure is depicted as an 
information flow on 4 Chart 3 on page 8. - 

- Document searches are organized around the variables listed in 
Chart 1 anjl its subsequent versions ^and are initiated according to 
priorities- negotiated between the Alaska Office of Planning and Research 
and^the contractor. 

Once an item is located, }t is given a number, a short title and 
assigned to a reviewer (Step 3). Most items- are journal articles or 
published reports.; but other types of documents can be accommodated as 
well. Primary sources are preferred over secondary sources, but in the 

early stages of the project secondary sources are necessary and useful, 
i 

The analysis of each document (Step 3) is conducted using the format 
called for in the Item Report form, a completed sample of which appears on 
pages 11 and 12. Most of the headings are self-explanatory but a few \ 
deserve mention. Tor example, the notation % Skimmed , Rejected for Project 1 
Purposes, No Analysis , when checked, means that a document has been^scanned 
and determined not to be useful for any of the purposes of the project, and 
no further analysis will be conducted. Immediately below is the notation 
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CHART 3 



RESEARCH ON SCHOOL EFFECTIVENESS PROJECT 
Information Flow 



r* Document Search: 



NtfREL INFO 
CENTER 



.0 



ALASKA OFFICE OF 
PLANNING AND RESEARCH 



\ 



OTHER 
SOURCES 



i 



Research on School Effectiveness Project 
NWREL 



i 



Assigns I ten No. and /T\ 
Short Title, Give to Reviewer \zJ 

i 



Review and Analyse ^ Documents tc 

Using Item Report Form, (3J ^ Backup Fllc 

Assign Descriptors 



Documents to 



I 

WANG 

I 



Review, approve, 
or correct 



i 



XEROX multiple copies, 
(one per descriptor) 
(plus one for backup file) 



fx 0 

0 

© 



Distribute e Category Files, by Descriptors 

& & . ■■ g 



^ Monitor Category Files and 
Maka Deciaion to Render Report 




Assign to Writer 

I 

Write Report in ST\ 
•Standard Format Vtii/ 

i 

WANG (n) 

1 

Review, Approve, or Correct 

• I ' 

Transmit to Alaska /~N 
Office of Planning and \JJ 
Research, for Review, 
Approval, or Corrections 



I 



JO 



Submit to Alaska 
Task Force on n^j 
Effective Schooling 



BEST COPY AVAILABLE 



Relevant Irrelevant _ for Present Summary Report . This notation is 

not checked until after the analysis is complete and a decision has been 
made to include or not include the information in the synthesis of a summary 
report (Step 10). Another notation which bears comment is Rating of Quality 
Stud y ; These ratings are made according to the criteria set forth in 
Standards for Rating Studies , which appears on page 13. 

After Item Reports are. drafted (Step 3), they are typed into the WANG 
word processor, printed, reviewed, approved or corrected and approved, and 
then xeroxed, one copy for £ach descriptor assigned plus one for a general 
backup file, and filed .by category (Step 7). These Categories are the 
same as the ERIC Descriptors used to identify the variables as first $et 
out in Chast 1, * 

The category files grow as Item Reports are completed . These file? 
are monitored by the projec* director, and when sufficient information has 
been collected, a decision is made to render a summary report <Step 8). 

The report is written (Step 10) in a standard format and typed on * 
the WANG word processor. The format for Summary Report by Category is * 
displayed on page 14. Once again, the headings on the form are generally 
'self-explanatory, but ^one requires comment. The heading, Decision Display , 
refers to a supporting form, a completed example of which appears on pages 
15 and 16. After all of the Item Reports in the file are examined, the 
writer identifies the issues and restates them as hypotheses. The Decision 
Display is used to assist in the analysis. The analysis is judgemental; 
there are no mathematical formulae applied or calculations performed. 
Aftei. analysis, the hypothesis is accepted or rejected and recast as a 
conclusion in -the Summery Report , Some Summary Reports address more than 



one issue, result in more than one hypothesis, and Decision Displays yit-ld 
* 

more than a single conclusion* 

^, « 
In short, 1 the Summary Reports are supported bv Decision Dis/la ys , 

which in Curn aire supported by copies of the Item Reports . 

The Summary Reports are reviewed and corrected andfoj approved and 
transmitted through ^the Alaska- Office of Planning and* Research m to the 
Governor's Task'Force on Effective Schooling (Steps 12, lj and 14)' . 

An important feature of this whole system is that it; us possible to 
challenge and replicate any report or part of a* report (Step 15). Bv ^ » 
following the dashed lin»s, it can be seen how the process can be re-entered 
at most any point. This might be done whenever major additional research 
becomes available o*- if the interpretation of aav document or gi^oup of 
documents is seriously questioned. Such challenges and changes could be % 
handled without redoing the, entire effort. This system provides a flexible, 
economical way to keep abreast of * knowledge about school effectiveness as 
it becomes available or is reinterpreted. It provides the educational 
community with a basis for establishing standards on a solid base of 
research knowledge, while recognizing that < hat is the best knowledge 
today could possably change in the future. 
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ITEM NUMBFR: 41* 
REVIEWER: K. Cotton 



LOCATION: NWREL Info. Cntr. Periodicals 
DATE REVIEWED: 11/I9/B0 



CITATION: Sarth, Richard. Home-based Reinforcement of school' behavior : a 
review and analysis, review of Educational Research. 1979 # 49, 
436-458. 1 : — 



DESCRIPTORS: parent Participation 



ShORT TITLE: Barth, 1979. Home>Based Reinforcement 
SK T >iED, REJECTED FOR PROJECT PURPOSES, NO ANALYSIS 
KZLEVANT IRRELEVANT FOR PRESENT PURPOSES 



PRIMARY SOURCE SECONDARY SOURCE X 



DISSERTATION ABSTRACT 



5 (Strong) 



RATING OF QUALITY OF STUDY (for project purposes): 
;weak) 1 2 3 (4J 

BRIEF DISCUSSION OF RATING: 

This is an important secondary souc.ce with clear findings. 

« 

SYNOPSIS* L 

i 

This is a review of twenty-fout studies of home-based reinforcement of school 
behavior as a method for motivating -behavioral change. The review gives 
special attention to types of consequences employed and methods of gaining 
parental involvement. The studies reviewed have examined home-based 
reinforcement programs tor children in group homes, in special classes «and 
with entire mainstream and special classe*. The programs studies employed a 
wide range of reilff orcers, including food, praise, privileges, etc., and the. 
method of involving parents ranged from intrequent notes/instructions to 
aetailed ana frequent par ent-schtpol staff conferences. Outcome goals of the 
programs studies included reduced discipline problems, increased academic 
achievement (measured variously) and others. 
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ITEM NUMBER: 



41 



SHORT TITLE: Barth, j 1979 

Home-Based Reinf arcement 



RESEARCHER'S FINDINGS: 

Nearly all" kinos of home-based reinforcement studies reviewed indicated that 
the programs produced some desired outcomes, regarfiless of student 
characteristics, type of reinforcer, kind of parent involvement procedure or 
kind of school setting. Person-to-person communication between parent and 
teacher appeared to result in the most successful home reinforcement programs. 



RESEARCHER 1 S CONCLUSIONS: 

Home-baaea reinforcement etrorts are effective an supporting a variety of 
school-initiated behaVior change activities. Care must be taken in phasing 
out such reinforcement etforts, as some studies lnoicated a return to less 
desirable behavior when they were withdrawn. * 



REVIEWER'S NOTES AND 



^MME 



Nrrs: 



An extensive bibliography is included in the Parent Participation backup file. 
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STANDARDS FOR RATING STUDIES 



Rating Description 

5 Outstanding study —good design and methodology, appropriate 

and adequate sample/population, extensive and appropriate 
data, clear-cut findings which allow for confident generalizations 
and conclusions. 



ERJC 



Very good^study — may have minor problems with design, 
methodology, sample/population or data, but findings are 
reasonably clear and strongly suggestive. Conclusions 
drawn are reasonable. 



Good study — may have problems with design, nethodology , 
sample/population or data, but findings are clear enough 
to allow some general conclusions. ? 



Fair study — may have major problems with certain aspects of 
design, methodology, sample/population or data, but there 
are some findings which are clear enough to- allow certain 
limited conclusions. This is the lowest quality sturdy 
acceptable for inclusion. 

Poor study — major problems with design, methodology, sample/ 
population or data. Findings unclear or unjustified, no 
useful conclusions may be dra%m. Study not acceptable for 
inclusion. 



X 
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RESEARCH ON SCHOOL EFFECTIVENESS PROJECT 



Topic ; 



Author: 



Date 



Overview : 



Format for 
Summary Report by Categpry 



Findings: (Including discussion of conflicting findings) 



Conclusions : Y 

v r > - 

Recommendations: (This could include cases where the recommendation 
would be that there be no standard) 



Decision Display(s) 



Bibliography: (Item Nos. and citations) 



r 



File of Item Repor 



t$^ (By item number) 



14 ■ 
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PARENT PARTICIPATION 
Decision Display 



Restatement of issue as a hypothesis: 



Programs which involve parents in the 1 education of th%ir preschool and^elementary 
school children havfe a positive effect on the academic achievement of those 
children. 



Item 
Number 



Short Title 



Quality Rating 
ot Stuay 

; t 



Items^which tend to support hypotheses : 

\ 



40 

41 

51 
39 
J3- 

46 



37 
30 



52 



32 



Bronf enbrenner , 1974. Early Intervention 

Research Synthesis " 
Barth, 1979. Home-Based Reinforcement (4) 
henderson & Swanson, 1974./ Parent Tutoring (4) 



[5] 1 26 studies support) 



©•Neil, 1975. Parent Tutoring in Reading 
Austin, et^ al . , 1977. Parent Involvement 

ir. Child Instruction, Low SES 
Lee, 1978. Parent Participation in 

Dental Programs 
Gillum, et al . , 1977. Parent Involvement 

in Performance Contracts 
Hickey, 1977. Modifying Work Habits 
Irvine^ et^ £l . , 1979. Parent Participation [2] 

in Preschool Programs 
McKinney, 1975* Parent Tutoring in (2) 

Reading and Math 
Seiber^t, et 1979. Parent School [2] 

Communication 



14] • 

(3) (18 studies support) 
13) 
12] I 
12] 



Items which tfcnp to aeny hypothesis : i 

41* Barth, 19.79. Home-Based Reinforcement g 14] 

68 Gabel,^et al.« 1977. Parent -Teacher 14] 

Communication in LD Classes 



Items whifh are inconclusive regarding the hypothesis : 

38 Bedford-Stuyvesant , 1978. Family Education |2J 

70 Heisler Crowley, 1969. Parent % 12] 

Participation in a Depressed Area 

50 Olmsted, 1979. Parent Teaching Behavior* [2) 



ERLC 



17 



15 



Items which were excludeo becAse they were weak ; 

28 Imber, et al . , 1979, Parent frfcrticipation [1] 

69 * Quisenberry, 1980. Home-Oriented Preschool ]l) 
Program 

45 Robinson, 1978. Citizen Aavisory , (l] 

Committees 



Items which were .ettcludea because they were pudged to be irrelevant to th is 
hypothesis : 

29 McLaughlin, et al . , 1980. Increasing Farent Participation 

of Native Americans * v 

42 Rich, et al., 1979. »Non Deficit" Moael for Parent Involvement 

44 Tudor, 1977. Teacher Attitude Toward Parent' Involvement 

63 Hertford Public Schools, 1974. Heaastart Parent Participation 

64 . Townes, et al., 1979. LD Parent Participation 

65 Lincoln, et al., 1975. Parent Participation/Math "Growth Sessiops" - 

66 Lucas & Lusthafcs, 1978. Parent Part lcipat ion, \lementary. vs . Secondary 

67 Bridge, 1976. J Effective Parent Participation 
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APPENDIX 



< 



CATEGORIES 
(Suggested by Design Group) 



Class' size 
School size 

Use of dif fererjtialed staff 
Teacher certification 

Teacher competence in content 1 • , ^ ' 

Teacher training in pedagogy * 
Teacher background 

Role of principalis instructional leader ? 
Facilities 

.Finance n— 
Nutrition 

f 

Vse of homevdrk ^ 
Time on task 

Instructional practices K 
Pupil grouping ^ « 

Match between m^terials/equipment/technology and pupils 
Language background of students 
Use of bilingual approaches 
Use of teacher evaluation 

— s 

Staff development efforts 
School climate 

Administrative policies and practices 

Curriculum ^ \ 

Student activities * - 

P a r en t / c ommun i t y i n vo 1 \ emen t 
Attendance 
Preschools 

Home environment * 



